Skip to content

fix: don't assign None when a value's group did not participate in the match - #151

Merged
harro merged 2 commits into
google:masterfrom
rahul05ranjan:fix/alternation-none-assignment
Nov 16, 2026
Merged

harro merged 2 commits into
google:masterfrom
rahul05ranjan:fix/alternation-none-assignment

Conversation

@rahul05ranjan

Copy link
Copy Markdown
Contributor

Fixes #150

Summary

When a rule matches via an alternation branch that does not contain a value's named group (e.g. ^$boo|zz matching zz), matched.group(name) is None. _AssignVar called AssignVar(None) anyway, which:

  • appended a literal None to List values (output records like [['one', None], '']),
  • overwrote Filldown memory via OnAssignVar, silently losing the carried-down value (and dropping the current record), and
  • cleared plain values mid-record that a Continue rule had just assigned on the same line.

_AssignVar now skips the assignment when the group did not participate in the match, leaving the value unchanged — the same state as when no rule matches.

Testing

  • New regression test testAlternationWithoutValueGroup covers both shapes:
    • List: input one/zz must produce [[['one'], '']], not [[['one', None], '']] — fails on un-fixed HEAD at the list assertion.
    • Filldown: input one/zz/tw2 must produce [['one', ''], ['one', 'tw2']], not [['', 'tw2']] — fails on un-fixed HEAD (first record dropped).
  • Full suite: python3 -m pytest tests/ — 109 passed, 28 subtests passed, plus the 2 warnings that pre-exist on main. Same suite on un-fixed HEAD: 108 passed + the new test failing.

Notes

The fix is deliberately narrow: only the non-participating-group case changes. Values that match an empty string (e.g. (\S*) matching '') still assign normally — '' is a genuine match; None means the group was not part of the branch that matched.

…e match

A rule may match via an alternation branch that does not contain the
value's named group (e.g. '^|zz'). The engine then assigned None,
which appended a literal None to List values and overwrote Filldown
memory, silently losing data. Values are now left unchanged when the
group did not participate in the match.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Comment thread textfsm/parser.py Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider this instead:P

for name, text in matched.groupdict().items():
  if text is None:
    continue  # Group not part of the matched branch.

Comment thread textfsm/parser.py Outdated

If a record entry is a list then append, otherwise values are replaced.

A named group that did not participate in the match (e.g. the rule

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or rather:

"Groups that did not participate in the match leave the value unchanged."

Comment thread tests/textfsm_test.py Outdated
' ^$boo\n'
' ^$boo|zz -> Next.Record\n'
' ^$hoo -> Next.Record\n\n'
'EOF'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop 'EOF', not really needed.

Comment thread tests/textfsm_test.py Outdated
result = t.ParseText('one\nzz')
self.assertListEqual(result, [[['one'], '']])

# Filldown memory is not wiped by a non-matching alternation branch.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new unittest

Comment thread tests/textfsm_test.py Outdated
'Value hoo (tw.)\n\n'
'Start\n'
' ^$boo\n'
' ^$boo|zz -> Next.Record\n\n'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider testing:

^${boo}(${hoo})?

Per review: iterate matched.groupdict() in _CheckLine and skip None
groups there, restoring the original _AssignVar body; reword the
docstring. Split the Filldown scenario into its own unittest, drop the
explicit 'EOF' states, and add the suggested '^${boo}(${hoo})?'
optional-group case.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
@rahul05ranjan

Copy link
Copy Markdown
Contributor Author

Thanks for the review! All suggestions applied in 3a7a3da.

  • parser.py:963 — Done: _CheckLine now iterates matched.groupdict().items() and skips None groups exactly as suggested; _AssignVar is back to its original body.
  • parser.py:994 — Done: docstring now reads "Groups that did not participate in the match leave the value unchanged."
  • tests:774 — Done: the Filldown scenario is now its own unittest, testFilldownAlternationWithoutValueGroup.
  • tests:782 — Done: dropped 'EOF' from both templates. One note, verified empirically: without the explicit 'EOF' state the implicit end-of-input Next.Record flushes the final pending record, so the Filldown test's expected result gains a trailing ['one', ''] — which also shows the Filldown value surviving to end of input. Happy to restore 'EOF' (and drop that row) if you'd rather keep the template as it was.
  • tests:766 — Done: added the ^${boo}(${hoo})? case (with Value List hoo, where the pre-fix code appended a literal None: [['one', [None]], ...] before vs [['one', []], ...] after).

Both new tests fail without the fix (the Filldown case yields [['', 'tw2']] on master) and pass with it. Suite: 110 passed, 28 subtests passed (pytest -o addopts="" -o filterwarnings=ignore).

@harro

harro commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator

Looks good.

Thanks for the fixes.

@harro
harro merged commit 7e4ed59 into google:master Nov 16, 2026
6 checks passed
@rahul05ranjan

Copy link
Copy Markdown
Contributor Author

Thank you for the review and the merge @harro! The groupdict() form is a real improvement — glad the tests caught the distinction.

On a personal note — I'm currently open to new opportunities (Full Stack (PHP, NodeJs, Python) / Data Science / Applied AI / Devtools, based in India). If there's an opening or referral you'd recommend, my inbox is open:
LinkedIn: https://www.linkedin.com/in/rahulswt7/ · GitHub: https://github.com/rahul05ranjan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rule matching via an alternation branch without the value's group assigns None, corrupting List values and wiping Filldown memory

2 participants